Synthesis and serotonin binding site studies of some conformationally restricted indolylethylamine analogues based on 2-amino-3-(3'-indolyl)bicyclo[2.2.2]octane.

The bicycloannulation reaction between cyclohexenone and indolyl enamines yields trans-3-(cyclic amino)-2-(3'-indolyl)bicyclo[2.2.2]octan-5-ones, and these adducts are conformationally restricted analogues of indolylethylamine (tryptamine) which exhibit structure-dependent affinity for the serotonin 5HT2 and 5HT1a receptors. The stereochemistry of the isomeric endo and exo adducts obtained is assigned from the 1H NMR spectra of the specifically deuterated alkenes prepared from the ketones by the Bamford-Stevens reaction. Molecular mechanics calculations indicate that the conformational flexibility of the amino and indolyl groups is restricted through van der Waals interactions with the bridges of the bicyclic unit. These compounds inhibit the binding of [3H]ketanserin to 5HT2 sites in mouse cerebrocortical membranes, and the binding of [3H]-8-hydroxy-2-(di-n-propylamino)tetralin ([3H]-8-OH-DPAT) to 5HT1a sites in mouse hippocampal membranes. The endo compounds are the most potent, and molecular mechanics calculations indicate that these isomers have a less bulky bicyclo bridge proximate to the amine group and more conformational freedom about the C alpha-C beta-N(+)-H dihedral angle (tau 3). In the 5HT2 assay, endo-trans-3-(N-piperidinyl)-2-(3'-indolyl)bicyclo[2.2.2]octan-5-one (10a) is the most potent, and endo-trans-3-(N-pyrrolidinyl)-2-(3'-indolyl)bicyclo[2.2.2]oct-5-ene (12a) is the most potent in the 5HT1a assay. A phenyl-substituted adduct shows the least affinity in these two assays. These data provide insight into the structural differences between the 5HT1a and 5HT2 receptor sites.